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MTie  Military  Message  Experiment  (MME)  was  designed  to  evaluate  the  utility  of  user-oriented 
message  processing  systems  in  a  military  environment  and  to  aid  in  determining  the  feature*  useful 
in  such  a  system.  The  experiment  is  a  cooperative  effort  betweenthe  Commander-in-Chi*  cific, 
the  Navy,  and  the  Defense  Advanced  Research  Projects  Agency.  To  conduct  the  experiment,  a  PDP 
10- based  system  was  installed  at  CINCPAC  Headquarters  for  use  by  a  portion  of  the  Operations 
Directorate.  The  message  processing  functionality  was  provided  by  SIGMA,  a  program  written  by 
the  Information  Sciences  Institute  of  the  University  of  Southern  California.  It  was  supported  by  the 
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20.  Abstract  (Continued) 

TENEX  operating  system,  and  the  user  terminals  were  modified  HP-2649A  CRTs. 

—  Th^MME  tysten^was  designed  to  give  the  user  the  capability  to  handle  his  message  traffic 
(both  incoming  and  outgoing,  formal  and  informal)  on  the  system.  The  system  enforced  multilevel 
security  rules  based  on  a  modification  of  the  security  kernel  model  developed  at  Mitre.  The  rule 
enforcement  was  not  rigorous  enough  for  certification,  but  it  was  sufficiently  rigorous  to  determine 
the  effects  on  the  users’  interactions  with  the  system.  Most  of  the  functions  needed  for  a  user’s 
message-related  tasks  were  provided  by  the  system:  message  distribution  and  redistribution, 
^electronic  read  board  ^  construction,  message  filing,  message  replies,  message  commenting  and 
'"chopping,*  and  message  release.  _ 

This  report  evaluates  the  MME  system  from  the  user’s  viewpoint.  The  CINCPAC  recommenda¬ 
tions  concerning  future  automated  message-handling  systems  arep>  ' 

a.  An  automated  system  must  be  reliable,  readily  available  to  all  who  need  to  use  it,  responsive, 
compatible  with  other  message-handling  equipment,  and  interoperable  with  its  support  message- 
transmission  system.  t-yr.t 

b.  It  must  be  designed  and  fielded  with  the  close  participation  of  the  user  community.  . 

c.  The  benefits  to  be  realized  from  an  automated  message-handling  system  are  not  of  such 
overriding  operational  significance  as  to  require  the  fielding  of  an  interim  system  that  does  not 
meet  the  user’s  needs. 
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1.  Introduction.  This  section  of  the  MME  Final  Report  will  cover  the 
period  from  April  1979  through  the  end  of  the  experiment  in  September 
1979.  It  will  discuss  the  daily  users'  perceptions,  the  simulated  CPX, 

a  variety  of  issues  that  were  not  included  in  the  final  MME  questionnaire 
but  are  deemed  to  be  significant  to  the  goals  of  the  experiment,  and 
recommendations  concerning  automated  message  handling. 

2.  Daily  Users'  Perceptions. 

a.  During  the  final  months  of  the  MME,  the  system  reliability 
Increased,  and  various  enhancements  were  made  to  the  software  which 
inproved  performance  and  made  the  system  more  beneficial  to  the  user. 

The  overall  feeling  of  most  users  toward  the  system  improved.  Especially 
noteworthy  were  the  attitudes  of  newly-arrived  personnel  who  had  not 
gone  through  the  system's  earlier  growing  pains;  they  were  very  satisfied 
with  it  and  had  a  positive  feeling  toward  automated  message  handling  as 

a  result  of  their  brief  exposure  to  SIGMA. 

b.  The  final  MME  questionnaire  was  concerned  with  user  views  on 
automated  message  handling,  independent  of  the  type  of  system  to  be 
used.  It  was  assumed  in  the  questionnaire  that  system  reliability  and 
ease  of  access  to  a  terminal  would  not  be  a  problem.  Fifty-three  ques¬ 
tionnaires  were  distributed,  with  38  responses.  For  each  of  the  major 
message-handling  categories  listed,  the  users  were  asked  various  questions 
including  whether  they  would  prefer  doing  that  function  with  an  automated 
system  (fully  reliable  and  easily  accessable)  or  a  manual  system.  Not 
all  respondents  answered  the  questions  in  every  category,  as  their  duty 
positions  determined  how  they  used  the  system.  Table  1  summarizes  the 
respondents'  preferences  for  automated  or  manual  message  handling. 

c.  The  user  comments  on  automated  versus  manual  message  handling 
were  much  more  positive  than  those  in  the  questionnaire  prepared  in 
March  1979  for  the  Mid-Experiment  Report.  Proponents  of  the  manual 
system  maintain  that  doing  a  particular  function  manually  is  "faster"  or 
"easier"  or  provides  "more  control,  more  flexibility,  and  more  selectiv¬ 
ity,"  or  has  "less  chance  of  error."  These  are  the  identical  reasons 
that  other  users  gave  for  preferring  automated  message  handling. 

d.  As  indicated  in  Table  1,  the  message-handling  function  which 
has  the  least  support  for  automation  is  message  review.  Even  having 
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Table  1 

Prefer 

Prefer 

Z  Prefer 

Function 

Automated 

Manual 

Automated 

MESSAGE  DISTRIBUTION 

17 

2 

89 

MESSAGE  REVIEW 

21 

11 

66 

MESSAGE  FILING 

25 

1 

96 

MESSAGE  RETRIEVAL 

32 

3 

91 

MESSAGE  CREATION 

22 

2 

92 

MESSAGE  COORDINATION 

14 

2 

88 

MESSAGE  RELEASE 

8 

1 

89 

. 
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experienced  the  improved  system  reliability  and  performance  during  the 
last  months  the  MME,  one-third  of  the  users  still  prefer  the  concept 
of  reading  messages  on  a  paper  copy.  (This  does  not  take  into  considera¬ 
tion  the  amount  of  effort  required  to  have  a  set  of  selected  messages  in 
an  action  officer’s  in-box  upon  his  arrival  in  the  morning.)  Reasons 
mentioned  for  preferring  a  manual  system  include  the  ability  to  determine 
rapidly  if  the  message  needs  to  be  read  at  all,  the  ease  of  scanning  the 
entire  message  or  skipping  back  to  recheck  a  portion  of  a  long  message, 
the  speed  of  getting  the  next  message,  and  the  proven  reliability  of  the 
paper  system. 

e.  Of  the  three  broad  message-handling  categories  (incoming 
message  processing, 'message  filing  and  retrieval,  and  outgoing  message 
processing),  the  users  shoved  the  greatest  preference  for  message  filing 
and  retrieval  (Table  1).  This  is  consistent  with  the  user  preferences 
reflected  in  the  Mid-Experiment  Report  questionnaires.  Informal  discus¬ 
sions  with  users  and  the  comments  on  the  final  questionnaire  indicated 
that  the  message  filing  and  retrieval  capability  would  be  the  one  most 
missed  upon  termination  of  the  MME. 

f.  The  users  shoved  a  strong  preference  for  the  concept  of  auto¬ 
mated  support  for  the  outgoing  message  process  (message  creation,  editing, 
coordination,  and  release).  Hie  release  function  was  significantly 
improved  with  SIGMA  release  2.23  (April  1979),  which  may  account  in  some 
part  for  the  favorable  change  in  user  perceptions  of  automated  processing 
of  outgoing  messages. 

3.  Simulated  Exercise  One  of  the  primary  goals  of  the  MME  was  to 
determine  the  usefulness  of  automated  message  handling  in  a  crisis 
situation. 


a.  The  MME  system  was  used  during  Exercise  Power  Play  in  March 
1979  as  a  backup  message-handling  system.  Because  of  low  system  reliabil¬ 
ity,  low  message  influx,  and  its  backup  role,  no  determination  could  be 
made  as  to  the  usefulness  of  an  automated  message-handling  system  in  a 
crisis  (or  exercise)  situation.  A  more  detailed  report  on  use  of  the 

MME  system  during  Exercise  Power  Play  is  contained  in  the  Mid-Experiment 
Report. 

b.  In  order  to  attempt  the  determination  of  automated  message 
handling  usefulness  in  a  crisis /exercise  situation,  a  simulated  Command 
Post  Exercise  (SCPX)  was  conducted  in  September  1979.  The  SCPX  was 
based  on  Exercise  Power  Play  79  (PP-79) ,  as  all  incoming  PP-79  messages 
had  been  retained  in  the  MME  archives. 

c.  The  concept  of  the  SCPX  was  to  use  the  MME  system  as  the  only 
message-handling  medium  for  the  Operations  Action  Group  (OAG) ,  which  is 
the  CINCPAC  Crisis  Action  Team  formed  to  coordinate  CINCPAC  actions 
during  a  crisis  or  exercise  situation.  (A  detailed  description  of  the 
OAG  is  in  the  Mid-Experiment  Report.)  A  controller  would  inject  the 


incoming  messages  into  SIGMA;  the  injection  rate  could  be  varied  to 
simulate  the  normal  peaks  and  valleys  of  message  arrival.  Each  OAG 
□ember  had  his  own  terminal  and  working  area  in  the  OAG  room.  A  printer 
was  also  available  in  the  OAG  room. 

d.  The  message  distribution  play  for  the  SCPX  differed  from  that 
of  PP-79  as  described  below. 

(1)  In  PP-79,  SIGMA  sent  all  incoming  messages  to  the  OAG  XO, 
who  reviewed  each  message,  made  distribution,  and  assigned  action  as 
appropriate.  The  XO  also  maintained  the  Status  of  Action  Log  and  the 
Significant  Events  Log. 

(2)  In  the  SCPX,  SIGMA  filed  all  messages  in  the  master 
message  file,  to  which  all  OAG  members  had  access,  and  forwarded  all 
messages  to  the  XO.  The  only  requirements  for  the  XO  were  to  review  the 
incoming  messages  and  assign  action  on  them  as  appropriate,  and  to 
select  items  for  the  Significant  Events  Log  (SEL). 

e.  The  Assistant  XO  (AXO,  not  a  player  during  PP-79)  monitored 
the  SEL  and  added  comments  to  each  item  filed  in  it  by  the  XO  so  that 
any  player  could  look  at  the  SEL  and  see  a  synopsis  of  the  significant 
events  without  having  to  display  the  message  itself.  The  player  had  the 
ability  to  display  and  read  the  entire  message  if  he  so  desired.  Thus 
someone  could  monitor  the  progress  of  the  crisis/exercise  on  a  terminal 
in  his  office  without  having  to  go  to  the  Command  Center  (OAG  location). 

f.  The  AXO  also  monitored  the  Status  of  Action  Log  (SAL).  SIGMA 
automatically  filed  a  citation  in  the  SAL  for  every  message  assigned  for 
action  by  the  XO,  as  well  as  in  the  action  officer's  pending  file.  When 
an  action  officer  completed  work  on  a  particular  message  or  set  of 
messages,  he  would  make  an  appropriate  comment  on  the  dtatlon(s)  in  the 
SAL  (to  which  all  players  had  access).  Periodically,  the  AXO  would 
review  the  SAL  and  transfer  all  completed  actions  to  the  Action  Completed 
file.  Thus,  the  SAL  contained  only  those  messages  on  which  action  was 
being  taken  or  had  been  recently  completed. 

g.  Action  officers  were  expected  to  perform  as  much  of  their 
message-handling  functions  as  possible  using  the  terminal,  but  were  free 
to  use  the  printer  as  they  desired.  Each  action  officer  was  expected  to 
review  the  master  file  on  a  periodic  basis  for  messages  relating  to  his 
functional  area.  Each  action  officer's  Alert-Selector  contained,  as  a 
minimum,  For-Action,  so  that  an  action  citation  from  the  XO  would  come 
to  the  Immediate  attention  of  the  action  officer.  An  action  officer 
could  retain  messages  of  interest  to  him  by  either  marking  the  message 
with  a  Keyword  of  his  choosing  or  by  creating  appropriate  files  for 
various  messages.  Both  methods  were  used  during  the  SCPX. 

h.  Action  officers  created  outgoing  messages  and  coordinated  them 
within  the  OAG  as  required.  The  message  was  then  sent  to  the  XO,  who 


me 


checked  it  and  simulated  its  release  by  sending  it  to  the  controller. 

The  controller  double-checked  the  message,  then  released  it  to  the  LDMX, 
which  provided  the  drafter  a  back-copy  through  SIGMA. 

i.  The  daily  CINCPAC  Situation  Report  (SITREP)  was  prepared  in 
the  following  manner: 

(1)  Each  action  officer  prepared  his  portion  of  the  SITREP  in 
a  Text  Object  name  Jx-SITREP-INPUT  (where  the  x  represents  the  action 
officer's  OAG  code). 

(2)  When  an  action  officer  had  finished  his  portion  of  the 
SITREP,  the  J31  representative,  who  was  responsible  for  the  overall 
SITREP  preparation,  would  GET  the  text  object,  VIEW  it  and  COPY  the  text 
into  the  appropriate  portion  of  the  In-Preparation  SITREP. 

(3)  Once  the  SITREP  was  completed,  it  was  processed  in  the 
same  manner  as  any  other  outgoing  message. 

j.  The  SCPX  demonstrated  that  an  automated  message-handling 
system  as  represented  by  SIGMA  is  usable  in  a  crisis/exercise  situation. 
The  SCPX  players  were  able  to  keep  up  with  the  incoming  message  flow 
(the  maximum  rate  of  message  input  was  30  per  hour).  They  were  able  to 
file  and  retrieve  messages  in  a  timely  manner.  They  were  able  to  draft, 
coordinate,  and  release  outgoing  messages,  including  the  SITREP.  The 
backup  system  devised  by  the  MME  staff  to  provide  support  in  the  unlikely 
event  of  system  failure  worked  satisfactorily.  However,  the  SCPX  did 
not  demonstrate  that  an  automated  message-handling  system  (as  represented 
by  SIGMA)  is  more  efficient  or  effective  than  a  manual  system  in  a 
crisis/exercise,  based  on  the  following  indicators  concerning  speed  of 
message  processing  and  quality  of  outgoing  messages. 

(1)  Message  Receipt.  Messages  arrived  at  each  player's 
terminal  during  the  SCPX  in  a  rapid  manner,  regardless  of  precedence. 
During  PP-79,  messages  of  Immediate  and  higher  precedence  were  delivered 
by  the  manual  system  to  the  XO  as  fast  or  faster  than  by  SIQ1A.  SIGMA 
delivered  Priority  and  Routine  messages  to  the  XO  about  30  minutes  ahead 
of  the  manual  system.  (The  difference  is  due  to  the  LIMX  sending  high- 
precedence  messages  directly  to  a  line  printer  in  the  Command  Center. ) 

A  principal  advantage  of  an  automated  system  would  be  the  capability  to 
distribute  messages  selectively  to  each  OAG  member,  thus  relieving  him 
of  the  need  to  sort  through  all  the  messages  to  find  the  ones  of  Interest 
to  him.  SIQIA's  capability  for  user-created  selectors  is  a  major  step 
in  this  direction;  unfortunately,  the  loose  standards  for  the  format  of 
a  military  message  do  not  allow  SIGMA  to  satisfy  this  message-selection 
requirement  completely.  Thus,  both  the  manual  and  automated  systems 
delivered  messages  to  the  OAG  in  a  timely  manner  (considering  the  message 
precedence)  and  in  sufficient  quantity.  The  rapid  delivery  of  messages 
by  an  automated  system  may  be  offset  by  the  ability  of  a  user  to  scan 
paper  copies  more  rapidly  than  the  automated  system  can  display  them. 
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(2)  Message  Filing  and  Retrieval.  In  the  manual  system,  the 
OAG  clerk  maintains  a  master  file  of  all  incoming  and  outgoing  messages, 
generally  in  time-of-arrival/time-of-dispatch  sequence.  SIGMA  delivered 
all  messages  (incoming  and  back  copies  of  outgoing)  to  the  master  file 
in  time-of-arrival  (at  SIGMA)  order.  Since  in  the  manual  system  each 
OAG  member  receives  a  copy  of  each  message,  he  can  create  files  by 
subject,  originator,  etc.  (extra  copies  of  paper  messages  are  easily 
reproduced  in  the  Command  Center).  SIGMA  allows  the  creation  of  files 
by  each  user  and  also  allows  each  user  to  mark  messages  in  his  copy  of 
the  master  file  by  keyword.  The  major  advantage  of  the  automated  system 
is  the  capability  for  retrieval  of  messages  on  multiple  parameters,  such 
as  user-assigned  keywords,  subject,  originator,  date-time  group,  or  any 
combination  thereof.  Another  advantage  is  that  each  user  has  ready 
access,  through  the  date  files,  to  any  previous  message  concerning  his 
area  of  Interest,  even  if  the  message  arrived  prior  to  the  recognition 
that  a  crisis  was  impending  or  the  start  of  an  exercise.  However, 
unless  the  user  makes  a  paper  copy  of  each  message  he  needs  for  reference 
when  composing  an  outgoing  message,  he  can  VIEW  only  one  reference  at  a 
time  while  displaying  the  In-Preparation  message.  This  makes  it  awkward 
if  there  are  several  messages  to  which  a  user  needs  to  refer  in  composing 
his  outgoing  message.  Overall  user  preference  is  for  automated  filing 
and  retrieval,  especially  if  a  printer  is  readily  available  for  production 
of  hard  copies  of  messages. 

(3)  Message  Preparation,  Coordination,  and  Release.  SIGMA 
provided  a  good  capability  for  the  preparation  of  outgoing  messages, 
especially  for  users  with  adequate  typing  skills. 

(a)  Users  did  not  have  to  compete  for  the  services  of 
the  OAG  clerk-typist  when  more  than  one  outgoing  message  needed  to  be 
typed,  as  has  been  the  case  in  the  manual  system.  The  capability  of 
copying  sections  of  text  from  an  object  in  the  view  window  to  the  In- 
Preparatlon  message  was  very  helpful,  especially  in  composing  a  SITREP. 

(b)  Coordination  of  an  outgoing  message  within  the  OAG 
was  an  easy  process  in  both  systems.  If  a  message  needed  to  be  coordina¬ 
ted  outside  the  OAG,  the  advantage  of  the  automated  system  Increased  as 
the  distance  of  the  coordinator  from  the  Command  Center  increased. 

However,  if  there  were  off-line  references  required  by  the  coordinator, 
the  speed  advantage  of  the  automated  system  was  lost.  The  great  majority 
of  OAG  messages  do  not  require  coordination  outside  the  Command  Center; 
thus,  manual  coordination  does  not  lag  appreciably  behind  the  automated 
system,  especially  if  off-line  references  must  be  obtained  anyway. 

(c)  Once  a  message  is  approved  for  release,  SIGMA  will 
get  it  to  the  LDMX  for  release  to  the  AUTODIN  network  faster  than  the 
manual  system.  Manual  messages  are  sent  by  pneumatic  tube  from  the 
Command  Center  to  the  Communications  Center.  The  manual  message  must 
then  be  read  by  the  OCR  equipment  and,  if  accepted,  processed  by  the  - 
LDMX  and  released  to  AUTODIN. 
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(d)  A  great  advantage  of  SIGMA  was  the  capability  to 
readdress  messages  and  release  them  rapidly.  This  was  one  of  the  most 
appreciated  features  of  the  automated  system. 

(4)  Quality  of  OAG  Product.  A  hypothesis  was  mad 2  by  one  of 
the  SCPX  evaluators  that,  if  an  automated  message-handling  system  were 
to  have  military  utility,  it  would  have  to  process  messages  faster 
and/or  contribute  to  a  better  quality  output.  The  determination  of  the 
relative  quality  of  OAG  output  between  PP-79  and  the  SCPX  would  require 
a  very  subjective  judgment  as  to  which  of  the  outgoing  messages  (PP-79 
vs.  SCPX)  in  response  to  a  particular  event  was  "better,"  however  "better" 
might  be  defined.  Even  if  a  determination  of  quality  were  made,  it 
would  be  difficult  to  ascribe  the  difference  in  quality  solely  to  the 
assistance  of  the  automated  message-handling  system  because  of  the 
widely  differing  circumstances  under  which  the  messages  were  produced, 
e.g. ,  different  players  producing  the  messages.  Thus,  any  conclusion 
with  regard  to  the  military  utility  of  an  automated  message-handling 
system  based  on  the  comparative  quality  of  outgoing  messages  would  have 
doubtful  validity  at  best. 

(5)  Summary.  Comparison  of  message  handling  in  PP-79  (manual) 
and  the  SCPX  is  inconclusive  in  determining  if  an  automated  system  is 
more  effective  or  efficient  than  a  manual  system. 

(a)  Messages  arrived  at  the  OAG  slightly  faster  in  the 
manual  system  for  high  precedence  traffic  and  much  faster  by  the  automated 
system  for  low  precedence  traffic;  however,  both  systems  delivered  all 
messages  in  time  to  meet  the  needs  of  the  players. 

(b)  Both  systems  provided  a  satisfactory  message  filing 
and  retrieval  system.  The  automated  system  was  faster  in  retrieving 
messages  that  were  received  prior  to  the  OAG's  convening.  Depending 
upon  the  way  the  files  were  set  up,  a  particular  message  could  be  found 
as  quickly  in  the  manual  system  as  in  the  automated  system. 

(c)  The  automated  system  was  faster  for  creating  outgoing 
messages,  especially  if  the  user  was  an  adequate  typist,  and  for  readdres- 
sals.  Coordination  required  about  the  same  time  in  both  systems. 

Release  is  faster  by  the  automated  system.  All  outgoing  messages  were 
processed  satisfactorily  by  both  systems. 

(d)  No  valid  comparison  of  the  quality  of  outgoing 
messages  produced  by  the  manual  and  automated  systems  can  be  made. 

4.  Miscellaneous  Issues.  The  system  designers  raised  some  issues 
concerning  various  aspects  of  automated  message-handling  systems  which 
were  not  addressed  in  the  questionnaires  completed  by  the  users.  This 
section  will  cover  these  issues  based  on  discussions  with  users  and 
germane  user  comments  in  the  questionnaires. 
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a.  Training. 


(1)  One  of  SIGMA's  features  was  on-line  assistance  and  training, 
consisting  of  feedback  to  the  user  when  SIGMA  could  not  execute  an 
instruction,  the  PROMPT  facility,  the  HELP  facility,  and  the  LESSON/EXER¬ 
CISE  facility.  The  MME  staff  included  a  full-time  trainer,  who  conducted 
periodic  Introductory  Lectures,  more  advanced  training  as  requested, 
special  classes  prior  to  certain  scheduled  MME  events,  e.g.,  the  Simulated 
CPX,  and  who  provided  on-call  assistance  when  users  were  unable  to 
perform  a  desired  task.  In  addition,  there  was  a  trainer  in  J34  whose 

job  was  to  ensure  that  each  user  was  capable  of  operating  the  system  at 
a  given  level  of  competence  for  hi3  style  of  use  (action  officer,  J301, 

Air  Desk,  etc.).  However,  the  J3  trainer  departed  in  mid-April  1979  and 
her  replacement  did  not  assume  the  MME  training  function. 

(2)  The  users  were  widely  divergent  in  their  views  concerning 
initial  training.  Some  favored  formal  instruction,  some  peer  training, 
some  self-training.  Most  indicated  that  early  "hands-on"  experience  was 
valuable  in  becoming  familiar  and  comfortable  with  the  system,  especially 
for  those  who  had  not  previously  interacted  with  a  terminal.  Since  the 
MME  system  was  not  used  as  an  exclusive  message-handling  system  within 
J3,  there  was  no  demand  for  everyone  involved  in  message  handling  to 
become  proficient  in  a  short  period  of  time  and  to  maintain  or  improve 
their  skills.  Those  who  were  interested  in  using  the  system  achieved  a 
high  degree  of  proficiency  and  developed  their  own  "tricks  of  the  trade" 
to  make  the  system  perform  for  them.  Those  with  low  interest  learned 
just  enough  to  be  able  to  "get  by."  For  example,  some  Air  Desk  officers, 
in  building  the  Readboard  (a  daily  collection  of  messages  deemed  to  be 
of  particular  interest  to  the  J3),  would  look  at  each  message  in  the 
Pending  file  in  sequence,  either  filing  it  in  the  Readboard  and  then 
deleting  it  from  the  Pending  file,  or  just  deleting  it.  Others  would 
use  Selectors,  the  Route  instruction,  and  other  more  sophisticated 
techniques  to  build  the  Readboard  in  much  less  time  than  the  straight¬ 
forward,  one-message-at-a-time  method. 

(3)  The  users  did  not  find  PROMPT  to  be  of  much  use.  The 
major  complaints  about  PROMPT  were  that  it  was  not  specific  enough  and 
that  it  did  not  give  an  actual  example  of  the  type  instruction  for  which 
the  user  was  seeking  assistance.  This  may  have  been  a  training  problem, 
as  most  users  did  not  realize  that  if  PRCMPT  gave  them  several  options 
for  an  ambiguous  instruction,  they  could  select  one  of  the  options  and 
receive  a  more  detailed  explanation  of  that  particular  instruction  form. 
Also,  most  users  probably  did  not  really  understand  that  SIGMA's  inter¬ 
pretation  of  an  instruction  was  situation-dependent,  so  that  the  same 
partial  instruction  could  be  interpreted  differently,  depending  on  the 
user's  current  state. 

(4)  The  users  also  did  not  find  HELP  to  be  of  much  use. 

Using  it  appeared  complicated,  and  the  same  information  could  be  gotten 
just  as  quickly  from  the  reference  manual.  However,  HELP  was  updated 
more  frequently  than  the  reference  manual,  which  made  it  valuable  for 
assistance  with  newly  introduced  commands. 
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(5)  The  LESSONS/EXERCISES  were  not  very  well  received,  princi¬ 
pally  because  they  were  verbose.  The  concept  is  good,  but  a  more  concise 
approach  needs  to  be  taken.  However,  the  exercises  were  appreciated,  and 
there  were  several  comments  that  there  should  be  more  of  them,  especially  in 
the  later  lessons  which  address  more  sophisticated  system  use. 


(6)  In  training  a  new  user,  a  balance  must  be  struck  between 
formal  classes  and  self-teaching  using  both  on-line  and  off-line  material. 
Refresher  training  should  be  made  available  for  those  who  use  the  system 
intermittently.  A  strong  effort  must  be  made  to  inform  users  of  new 
system  features,  instructions,  or  procedures.  Much  training  can  be  done 
by  a  student's  office  mates;  however,  this  presents  the  possibility  of 
bad  habits  and  poor  practices  being  passed  along. 

b.  Editing. 

(1)  The  editing  of  various  fields  of  an  In-Preparation  message 
was  straightforward  and  simple.  The  precedence  field  was  set  to  ROUTINE  by 
SIGMA  when  the  initial  In- Preparation  AUTODIN  message  form  was  displayed  to 
the  user.  This  could  lead  to  the  user’s  overlooking  the  precedence 
field  if  he  were  in  a  hurry,  resulting  in  a  message  being  transmitted 
with  a  ROUTINE  precedence  when  it  should  have  been  FLASH.  For  this 
particular  field,  it  would  be  better  to  make  the  precedence  a  parameter 

of  the  CREATE  instruction  or  to  draw  the  user's  attention  to  the  field 
with  a  prompt  of  some  nature.  This  also  applies  to  the  READDRESS  instruc¬ 
tion,  in  which  SIGMA  established  the  readdressal  precedence  as  ROUTINE, 
regardless  of  the  precedence  of  the  original  message. 

(2)  Editing  of  text  was  easy,  but  there  were  certain  editing 
capabilities  that  would  have  made  drafting  simpler,  e.g. ,  a  capability 
for  varying  the  line  spacing  so  that  a  draft  could  be  double  spaced 
without  having  to  physically  remove  the  carriage  returns  for  the  final 
version,  or  a  capability  to  capitalize  large  blocks  of  text.  Many  users 
had  trouble  with  SIGMA's  formatting  procedures,  although  a  modification 
permitting  the  user  to  delimit  the  text  he  wanted  formatted  was  a  substan¬ 
tial  improvement. 

(3)  A  user  capability  for  editing  objects  suet  as  Directories 
and  Files  (to  control  access,  for  example)  would  have  been  useful.  A 
very  desirable  capability  would  be  user  editing  of  Selectors.  Some 
users  had  selectors  of  considerable  length,  and  it  became  time-consuming 
to  re-do  the  entire  selector  when  desiring  to  delete  a  particular  specifi¬ 
cation. 

c.  Office/Individual  Identification.  The  major  problem  for  most 
users  was  not  the  double  log-on  requirement  (although  a  few  complained  about 
it) ,  but  rather  that  SIGMA  would  not  let  a  user  who  was  logged  on  as  an 
office  code  and  identified  as  himself  get  at  messages  (e.g.,  For-Chop) 

in  his  own  Mypending  file.  The  restrictions  placed  on  users  by  SIGMA 


were  very  vexing  Co  many.  At  Che  minimum,  every  user  should  be  able  to 
have  GET  privileges  on  his  office  code  objects,  e.g. ,  all  personnel 
assigned  to  J342  should  be  able  to  get  any  J342  object  while  logged  on 
as  an  individual  (unless  a  particular  object  had  specifically  been 
restricted).  This  could  be  expanded  further  upward,  e.g.,  J342  could 
GET  any  J34  object,  etc.  Every  user  should  be  able  to  access  and  work 
with  all  entries  in  his  personal  objects  at  any  time,  as  long  as  he  was 
properly  identified  to  the  system. 

d.  Experimental  Nature  of  System.  The  fact  that  the  system  was 
primarily  an  experiment  and  not  fully  operational  seemed  to  escape  many 
of  the  users,  who  became  irate  when  the  system  was  down  for  various 
reasons.  Their  reaction  is  not  unjustified,  as  their  participation  in 
the  MME  caused  an  extra  workload  and  at  times  resulted  in  having  to 
restart  work  which  was  almost  completed.  Every  effort  was  made  to  keep 
the  system  available  to  the  user  and  to  announce  scheduled  downtime  well 
in  advance.  However,  the  experimental  nature  of  the  system  permitted, 

and  often  encouraged,  user  feedback  of  great  value  to  the  system  designers. 
Many  system  enhancements  were  implemented  at  user  request,  such  as 
ROUTE,  SORT  (a  file  by  DTG) ,  controlled  formatting,  etc.,  and  were 
greatly  appreciated. 

e.  Internal  Messages.  SIGMA's  capabilities  for  passing  informal 
notes  and  formal  CINCPAC  memoranda  were  useful  but  not  essential.  The 
telephone  or  a  personal  visit  (depending  on  the  distance  between  offices) 
were  equally  satisfactory  for  informal  discussions.  There  was  rarely 
any  requirement  for  memos  to  be  created  and  delivered  at  electronic 
speed;  editing  and  re-writes  are  equally  easy  on  various  word-processing 
machines.  If  the  MME  had  extended  to  other  sites  outside  of  CINCPAC  HQ, 
the  capabilities  for  notes  might  have  been  much  more  appreciated,  espec¬ 
ially  if  the  material  under  discussion  were  classified.  A  desirable 
feature  would  be  a  direct  liak  between  terminals,  rather  than  the  proces¬ 
sing  of  notes  as  such.  An  advantage  to  the  use  of  SIGMA  notes  is  the 
keeping  of  an  accurate  record  of  what  was  said.  If  the  capability  for 
notes  and  memos  can  be  included  in  an  automatic  message-handling  system 
without  significant  cost  in  funding  and/or  system  performance,  it  could 
save  funds  that  would  otherwise  be  spent  on  word-processing  equipment. 

f.  Message  Distribution. 

(1)  The  ultimate  goal  in  message  distribution  should  be 
automated  distribution  to  a  particular  office  code  or  individual,  based 
on  a  user  pre-defined  set  of  criteria.  The  SIGMA  selector  attributes 
(DTG,  originator,  subject,  etc.)  are  a  good  start  iu  this  direction.  If 
the  automated  system  sent  each  message  to  a  user's  Pending  File  based  an 
the  user's  criteria,  it  would  save  a  great  amount  of  effort  that  was 
expended  by  J301  during  the  MME  in  distributing  messages.  This  could  be 
done  on  a  message- by-message  basis  or  when  the  user  logs  on.  Ideally, 
the  automated  system  would  make  a  full  text  search  to  determine  any 
match  with  a  list  of  user-supplied  keywords,  as  well  as  using  other 
message  parameters  (subject,  originator,  etc.). 
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(2)  The  automated  system  should  be  able  to  assign  Action  or 
Cognizance  to  a  particular  office  code,  based  on  user  pre-determined 
criteria.  All  criteria  used  for  message  distribution  should  be  inter¬ 
actively  modifiable  by  the  user,  with  appropriate  control  on  modifying 
the  Action/Cognizance  assignment  criteria. 

(3)  The  user  should  also  be  able  to  "pull"  messages  from  a 
particular  file  or  files,  based  on  a  set  of  user-specified  criteria,  as 
well  as  having  the  system  "push"  messages  to  him  as  described  above. 

(4)  The  system  should  be  able  to  send  messages  for  which  it 
cannot  identify  an  Action/Cognizance  assignment,  or  any  internal  addres¬ 
see,  to  a  service  position  for  resolution. 

(5)  As  mentioned  previously,  the  loose  format  standards  of 
the  current  military  message  make  it  more  difficult  for  an  automated 
system  to  accomplish  the  above  in  a  reasonable  manner.  Requiring  message 
creators  to  specify  a  SUBJECT  line,  a  list  of  KEYWORDS  contained  in  the 
message,  or  the  office  code  of  the  intended  receiver,  as  examples,  would 
greatly  assist  the  automated  system  designer  in  this  area. 

g.  Message  Coordination.  .The  Coordinate  function  as  implemented 
by  SIGMA  was  usable  but  requires  simplification.  This  is  one  of  the 
areas  which  caused  a  great  deal  of  user  dissatisfaction.  Release  2.2, 
implemented  in  January  1979,  greatly  improved  the  Coordinate  function,  but  it 
still  remained  complicated  and  hard  for  some  to  understand.  There  were 
a  number  of  factors  that  caused  user  unhappiness  with  the  automated 
coordination  cycle:  coordinator  not  on  the  system;  coordinator  not 
logged  on;  key  references  not  on-line;  only  one  on-line  reference  viewable 
at  a  time;  face-to-face  meeting  needed  with  coordinator  due  to  complexity 
or  controversially  of  message;  too  many  versions  of  the  message;  and 
difficulty  in  rapidly  determining  the  chop  status  of  each  coordinator. 

Some  of  these  problems  could  be  solved  by  development  of  better  opera¬ 
tional  procedures  to  integrate  the  manual  and  automated  systems.  Face- 
to-face  coordination  will  always  be  required  on  some  messages,  but  this 
does  not  prevent  the  initial  transmission  of  the  message  to  the  coordina¬ 
tor  via  the  automated  system,  nor  does  it  preclude  the  message  being 
displayed  during  the  face-to-face  session  and  coordinated,  edited,  or 
released  at  the  end  of  the  meeting.  The  status  of  each  coordinator's 
chop  could  be  displayed  on  the  drafter's  citation  (file  entry)  as  well 
as  in  the  message,  so  that  he  can  readily  determine  chop  status  without 
having  to  display  or  view  the  message  and  get  to  the  chop  field  at  its 
bottom.  Having  only  two  versions  of  the  message  (the  drafter's  original 
and  one  other  on  which  each  coordinator  could  make  changes/comments) 
would  simplify  the  drafter's  problems  in  having  to  refer  to  so  many 
different  versions  when  there  are  a  large  number  of  coordinators. 

Automated  coordination  becomes  more  valuable  as  the  distance  between  the 
drafter  and  coordinator  increases,  especially  if  coordination  with 
another  headquarters  is  required. 


h.  Access  Control.  Some  aspects  of  access  control  have  been 
discussed  in  paragraph  c  above.  With  the  exception  of  certain  requirad 
restrictions  on  particular  messages  or  categories  of  messages  (LIMDIS, 
SPECAT,  EYES  ONLY,  SIOP,  etc  ),  there  may  not  be  a  requirement  to  prevent 
access  to  most  messages.  If  an  action  officer  has  a  project  partly 
completed  and  becomes  ill  or  otherwise  is  not  available,  another  officer 
will  have  to  assume  responsibility  for  the  project.  If  all  the  original 
action  officer's  work  and  references  are  non-accessible  to  anyone  else, 
the  substitute  will  have  a  difficult  if  not  Impossible  task  in  completing 
the  project.  Since  a  file  consists  only  of  pointers  to  messages,  there 
should  be  no  need  to  restrict  access  to  it  unless  the  information 
contained  in  the  File  Entry  itself  required  protection.  Likewise, 
Selectors  contain  only  system  attributes,  so  there  should  be  no  need  to 
restrict  access  to  a  Selector  unless  a  particular  subject  or  originator 
required  protection.  Text  objects  can  be  kept  private  by  creating  them 
in  an  Individual's  rather  than  an  office  account.  If  it  were  deemed 
necessary  to  have  restricted  access  to  an  object  or  comment,  the  default 
should  be  PUBLIC,  perhaps  with  a  prompt  to  ensure  the  user  understands 
the  access  specification. 

i.  Role  of  Paper.  It  is  mandatory  to  have  a  printing  capability 
as  a  part  of  an  AHHS.  There  will  always  be  situations  in  which  paper 
copies  of  messages  are  required.  High-level  executives  may  not  desire 
to  have  a  terminal  in  their  offices  or  to  use  a  CRT.  There  will  almost 
always  be  someone  who  doesn't  have  a  terminal  and  with  whom  a  message 
must  be  coordinated.  Many  persons  find  it  easier  to  read  Incoming 
messages  on  paper  copy,  as  previously  discussed;  incoming  messages  may 
be  electronically  routed  to  the  appropriate  office  and  read  boards  of  key 
messages  made  with  paper  copies.  After  entering  text  into  an  In  Prepara¬ 
tion  message,  it  can  be  much  easier  to  make  revisions  on  a  paper  copy 
and  then  enter  them  on  the  terminal,  especially  if  it  is  a  long  message. 
If  the  AMHS  is  such  that  each  action  officer  does  not  have  ready  access 
to  a  terminal,  he  might  want  to  keep  paper  copies  of  messages  with 
current  interest.  There  will  undoubtedly  be  a  requirement  for  a  back-up 
paper  filing  system  so  that  messages  will  be  available  in  the  unlikely 
event  of  a  catastrophic  system  failure. 

j.  Archiving. 

(1)  The  30-day  on-line  retention  of  most  messages  seemed  to 
be  satisfactory  and  was  far  better  than  the  15-day  retention  period 
prior  to  the  installation  of  the  dual-density  disc  drives.  Users 
should  be  given  the  option  of  marking  messages  or  text  objects  for  on¬ 
line  retention  after  the  normal  time  had  come  for  archiving  them;  this 
would  need  to  be  controlled  closely  to  prevent  abuse,  with  periodic 
reminders  given  to  users  to  check  their  files  for  messages  no  longer 
needed  on-line. 

(2)  The  speed  with  which  a  user  wants  an  archived  message 
retrieved  varies  from  instantaneously  to  "whenever  is  convenient."  A 
reasonable  time  would  be  5-10  minutes.  Users  could  help  the  system 
operator  by  being  able  to  specify  in  the  retrieval  request  how  rapidly 
they  need  the  message,  e.g. ,  immediately,  one  hour,  tomorrow,  etc. 


12 


(3)  Date  files  should  be  kept  for  at  least  one  year  and 
longer  if  file  space  permits.  Users  should  be  alle  to  search  over  a 
range  of  dates  when  looking  for  a  particular  message. 

(4)  The  user  should  be  able  to  control  the  archiving  of 
personal  files  and  text  objects,  again  with  appropriate  controls  to 
guard  against  a  build-up  of  unneeded  objects  in  the  system's  on-line 
storage.  There  would  be  some  files  that  a  user  would  have  continual 
need  for,  although  the  message  content  might  be  continually  changing. 

Other  files  that  pertain  to  a  particular  event  could  be  archived  shortly 
after  the  event's  completion.  The  user  should  be  able  to  specify  files 
and  text  objects  for  archive. 

k.  Critical  mass  of  users. 

(1)  One  of  the  major  problems  with  the  MKE,  as  it  was  set  up, 
was  the  small  number  of  terminals  that  could  ba  supported  with  satisfac¬ 
tory  system  response,  and  thus  the  number  of  J3  offices  that  participated 
in  the  experiment.  No  other  CINCPAC  staff  directorates  participated  in 
the  experiment.  Many  outgoing  messages  needed  to  be  coordinated  with 
non-participating  offices;  this  meant  that  coordination  had  to  be  accom¬ 
plished  both  on  the  MME  and  manually.  If  an  automated  system  is  to  be 
useful  to  a  headquarters,  it  must  be  available  to  all  offices  that  are 
participants  in  the  message-handling  chain  and  have  enough  terminals  in 
each  office  so  that  action  officers,  secretaries,  unit  chiefs,  etc.  can 
have  access  to  the  system  in  a  reasonable  amount  of  time.  It  is  not 
necessary  that  every  action  officer  have  his  own  terminal,  but  there 
probably  should  be  one  terminal  for  every  two  action  officers,  and 
certainly  one  terminal  for  every  three  action  officers.  Each  office's 
workload  should  dictate  the  number  of  terminals  required  for  that  office. 
Another  possible  method  of  terminal  distribution  would  be  to  have  termi¬ 
nals  at  the  branch  level  and  higher,  operated  by  a  clerk  or  secretary. 

All  output  for  action  officers,  branch  chiefs,  etc.  would  be  entered  bv 
tl-*  terminal  operator.  The  principal  advantage  of  this  method  of  terminal 
distribution  and  system  use  would  be  a  reduction  in  the  required  number 
of  terminals.  However,  this  method  would  loss  much  of  the  benefits  that 
accrue  from  the  Interactive  features  of  an  automated  system  to  which  all 
users  have  ready  access,  such  as  not  having  to  wait  for  the  terminal 
operator  to  finish  several  other  jobs  prior  to  doing  yours. 

(2)  It  is  not  necessary  that  an  action  officer  or  office  keep 
the  terminal  on  all  the  time.  If  a  user  has  a  message  to  coordinate,  he 
can  easily  call  the  office  with  which  the  message  is  to  be  coordinated 
and  advise  that  there  is  a  message  for  coordination.  The  System  Status 
feature  and  Alert-Selector  feature  of  SIGMA  provide  excellent  assistance 
to  the  drafter  and  the  coordinator  in  the  coordination  process. 

1.  Audit  Trail.  Not  many  users  were  aware  of  the  detail  in  which 
SIGMA  recorded  system  and  user  activity;  if  they  were  aware  of  it.  It 
did  not  inhibit  their  use  of  the  system.  As  mentioned  previously,  there 
is  not  much  emphasis  on  privacy  in  a  military  headquarters ,  except  for 


specific  categories  of  messages.  The  documentation  provided  by  SIGMA 
could  be  useful  to  both  system  designers  and  to  the  training  cadre,  who 
could  analyze  a  user's  style  of  operation  and  recommend  more  efficient 

ways  of  system  use. 

a.  User  View  of  System.  Most  users  found  the  system  easy  to 
learn  and  use.  The  concept  of  the  response  line  to  tell  users  what  the 
system  was  doing  or  not  doing  is  good.  The  English  language  command 
words  and  syntax  were  straightforward,  and  the  system's  capability  to 
expand  instructions  and  correct  spelling  usually  helpful;  at  times, 
however,  the  systea  would  interpret  a  partially  entered  command  differ¬ 
ently  froa  the  way  the  user  had  expected.  The  system's  interpretation 
of  coaaands  based  on  the  situation  at  the  time  the  comaand  was  Issued 
caused  some  minor  difficulties.  The  concept  of  opening  and  closing 
objects  bothered  some  users  who  wanted  to  have  more  than  one  of  the  same 
kind  of  object  open  at  a  time,  so  that  the  user  could  SH(W  an  object 
rather  than  having  to  DISPLAY  it  (SHOU  operated  auch  faster  than  DISPLAY). 
This  would  aaka  it  easier  to  refer  to  several  messages  when  drafting  an 
outgoing  aessage.  Two  aajor  beneficial  features  of  the  systea  were  the 
capabilities  for  allowing  the  user  to  recover  froa  mistakes  easily  and 
for  making  it  difficult  to  delete  an  object  erroneously  by  requiring 
confirmation  of  any  coaaand  to  do  so. 

5.  gaccmsgendat ions .  The  recommendations  concerning  future  autoaated 

message-handling  systems  aade  in  the  CINCPAC  section  of  the  Mid-Experiment 
Report  remain  valid.  The  following  points  are  reiterated: 

a.  An  autoaated  systea  must  be  reliable,  readily  available  to  all 
who  need  to  use  it,  responsive,  compatible  with  other  message-handling 
equipment,  and  interoperable  with  its  support  aessage-transmission 
systea. 


b.  It  aust  be  designed  and  fielded  with  the  close  participation 
of  the  user  community. 

c.  The  benefits  to  be  realised  froa  an  automated  message-hand ling 
systea  are  not  of  such  overriding  operational  significance  as  to  require 
the  fielding  of  an  interim  systea  that  does  not  aaet  the  user's  needs. 
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